Fluke MSE performance metrics
Gavin Fay
2022-06-26
Read in the results objects
These were created with the performance-metrics.R
script.
- summary of state metrics
## `summarise()` has grouped output by 'om', 'mp'. You can override using the
## `.groups` argument.
## `summarise()` has grouped output by 'om', 'mp', 'state'. You can override using
## the `.groups` argument.- calculate the relative metrics comparing medians
## Joining, by = c("metric", "om")- calculate the relative metrics
## Joining, by = c("metric", "isim", "om")proportion outperforming mp1
create table of all final summary metrics
create table of all final metrics
Metrics by OM & MP
## Warning: Removed 3 rows containing missing values (geom_point).
metrics across MPs for all oms
Boxplots of metrics for each OM
First define plotting cateogrical values
Base Operating Model
#### MRIP bias Operating Model
Distribution Shift Operating Model
State level metrics
metrics by state (base om)
change_cs
cs_per_trip
expense
keep_one
mulen_keep
mulen_release
ntrips
trophy
metrics by state (MRIP bias)
change_cs
cs_per_trip
expense
keep_one
mulen_keep
mulen_release
ntrips
trophy
metrics by state (distribution shift)
change_cs
cs_per_trip
expense
keep_one
mulen_keep
mulen_release
ntrips
trophy
how many states do better than MP1?
## `summarise()` has grouped output by 'om', 'mp', 'metric'. You can override
## using the `.groups` argument.
## Joining, by = c("om", "metric", "state")
## `summarise()` has grouped output by 'om', 'mp'. You can override using the
## `.groups` argument.## # A tibble: 168 × 4
## # Groups: om, mp [21]
## om mp metric nbeat
## <dbl> <chr> <chr> <dbl>
## 1 1 MP 1 change_cs 0
## 2 1 MP 1 cs_per_trip 0
## 3 1 MP 1 expense 0
## 4 1 MP 1 keep_one 0
## 5 1 MP 1 mulen_keep 0
## 6 1 MP 1 mulen_release 0
## 7 1 MP 1 ntrips 0
## 8 1 MP 1 trophy 0
## 9 1 MP 2 change_cs 8
## 10 1 MP 2 cs_per_trip 8
## # … with 158 more rows
## Joining, by = c("metric", "isim", "state", "om")
## `summarise()` has grouped output by 'om', 'mp', 'metric'. You can override
## using the `.groups` argument.## # A tibble: 1,296 × 5
## # Groups: om, mp, metric [144]
## om mp metric state prop
## <dbl> <chr> <chr> <chr> <dbl>
## 1 1 MP 2 change_cs CT 1
## 2 1 MP 2 change_cs DE 1
## 3 1 MP 2 change_cs MA 1
## 4 1 MP 2 change_cs MD 1
## 5 1 MP 2 change_cs NC 0.01
## 6 1 MP 2 change_cs NJ 1
## 7 1 MP 2 change_cs NY 1
## 8 1 MP 2 change_cs RI 1
## 9 1 MP 2 change_cs VA 1
## 10 1 MP 2 cs_per_trip CT 1
## # … with 1,286 more rows